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Problem Statement 




• The ISS is moving to as much ground control for robotics as 
possible in order to free up crew time (SPDM ops are 100% GC) 

* Performing robotic operations on ISS is a time consuming 
process (1 -3 days) 

* Because planning for robotics operations is a constraint driven 
process, there are few windows (1-3 days) available 

• This limitation of available windows is in conflict with our 
desire and need for an increasing number of robotic operations 
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Objective 



* The objective of this effort is to increase the efficiency of ISS 
ground-controlled robotics operations 


• Use a phased approach to steadily increase the necessary 
system infrastructure and increase confidence in the system in 
a fashion similar to how ground control was implemented 


* The successful outcome will provide a tremendous increase in 
robotics utilization for the life of ISS as well as provide the 
blueprint (and validation) for future exploration missions 
utilizing remote robotic operations 
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Premise 




* The MSS is existing hardware and software that is 
not easy to modify (both from a cost and technical 
perspective) 

* Intelligent ground control aids can be developed to 
improve and increase efficiencies for the GC 
operator 

* The primary sensor that is available to provide 
information and help close the loop is video 
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Development Phases 



• Phase 1 - Initial Aide Identification and Development 

• Joint development effort with ER and DX 

• Aide would provide additional/enhanced information to the 
operator, like digital pose estimation based on Natural Feature 
Image Recognition (NFIR) 

- Starting with one of the most time-consuming and most frequent 
ground operations - SPDM grasp of hardware fixtures 

• Initial implementation would be a standalone box, i.e. something 
that can be taken to a simulator or MCC and just plug a video feed 
into it. 



Pose 

Information 




Development Phases (cont.) 

• Phase 2 - Increased Aide Functionality 

• Aide could suggest MSS commands or perhaps even go so far as 
generate the commands, but not send them. 

• Begin fusing data from various camera views and MSS system 
data (i.e., control modes, joint position, etc.) 

• Ground retains full control over what commands get sent onboard 





Pose & 
Commands 
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Development Phases (cont.) 



• Phase 3 - Limited Autonomy 

• Aide generates and sends a subset of MSS commands perhaps 
limited to only non-motion commands or motion commands 
limited by distance to structure and TDRS coverage 

• More mature and increased data fusion capability 

• Ground still does “close quarters” commanding, and would still 
have the responsibility to monitor and be prepared to safe the 
system if necessary 



Pose & 
Commands 
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Development Phases (cont.) 



• Phase 4 - Full Autonomy 

• Aide generates and sends all MSS commands regardless of 
motion distance and operational complexity 

- No motion during Ku-band or S-band LOS 

• Full maturity and intelligence of data fusion capability 

• Ground would still monitor and would still be able to safe the 
system if necessary 
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Development Phases (cont.) 

• Phase 5 - Full Autonomy (On-Board ISS) 

• Aide generates and sends all MSS commands regardless of TDRS 
coverage 

- Stand-alone software would be needed onboard to continue during 
LOS periods (may require sub-phases to get to full implementation) 

- No changes to MSS software 

• Ground would still monitor when not LOS and would still be able 
to safe the system if necessary 
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Current Status 



• ER and MOD awarded funds for Phase 1 through the 
JSC IR&D for proof-of-concept development 


* ER developed NFIR capability based around MTC 
target and grasp operation using video from recent 
grasp operations 

• Initial results showed that the concept works as desired and 
that continued development is worthwhile 

• A real-time test in the MCC was done in June had very 
encouraging results 
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Enhanced Ground Control Demo 




User interface showing alignment positional error 


Green overlay provides visual feedback of tracking 
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Forward Work 



• Solution is being refined based on data collected during the real- 
time test in June 

• Work is starting on porting from Windows to Linux (MOD 
requirements in order to use the software in MCC) 

• Procuring MCC compatable frame-grabbing and processing 
hardware 

• Planning to expand the number of grasp target types supported 

• Only MTC target is in the database now - this is the most prominent 
target type and will be the “standard” target for all future hardware 

• Planning to expand capability to support operations beyond 
grasps like RPCM insertion and FRAM installs. 

* These are non-target based operations and will utilize surounding 
structure for the cueing information. 
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Forward Work (cont.) 



• Based on positive performance shown to date, a CR 
will be submitted for continued development of 
Phase 1 and Phase 2 

* Phase 1 activity would be an expansion of operations that 
can be used by ground control 

• Additional grasp targets 

• Non-grasp operations like FRAM installs and ORU insertions 

* Phase 2 would begin the building of a commanding 
capability fusing data from cameras and MSS data 

* Targeting demonstration for SpaceX-6 - 3 FRAMs up/3 
FRAMs down 

* Subsequent CRs would be submitted for the 
remaining Phases based upon the success of the 
proceeding Phase 



